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Evaluation of the Tennessee Voluntary Prekindergarten Program: 
Kindergarten and First Grade Follow-Up Results 
from the Randomized Control Design 

Executive Summary 

In 2009, Vanderbilt University’s Peabody Research Institute, with the assistance of the 
Tennessee Department of Education’s Division of Curriculum and Instruction, initiated a 
rigorous, independent evaluation of the state’s Voluntary Prekindergarten program (TN- 
VPK). TN-VPK is a full-day prekindergarten program for four-year-old children expected to 
enter kindergarten the following school year. The program in each participating school 
district must meet standards set by the State Board of Education that require each 
classroom to have a teacher with a license in early childhood development and education, 
an adult-student ratio of no less than 1:10, a maximum class size of 20, and an approved 
age-appropriate curriculum. 

TN-VPK is an optional program focused on the neediest children in the state. It uses a 
tiered admission process with children from low-income families who apply to the 
program admitted first. Any remaining seats in a given location are then allocated to 
otherwise at-risk children including those with disabilities and limited English proficiency. 

The evaluation was funded by a grant from the U. S. Department of Education’s Institute of 
Education Sciences (R305E090009). It was designed to determine whether the children 
who participate in the TN-VPK program make greater academic and behavioral gains in 
areas that prepare them for later schooling than comparable children who do not 
participate in the program. 

The current report is the second in a series that presents findings from this evaluation. The 
prior report described outcomes at the end of the pre-k year for the children in the 
Intensive Substudy sample who participated in TN-VPK in comparison to those who did not 
participate 1 . The present report summarizes the longitudinal effects of TN-VPK on 
kindergarten outcomes and those first grade outcomes that are currently available. At the 
end of prekindergarten, TN-VPK effects could be examined only on early achievement 
measures and teacher ratings of academic skills and behavior obtained at the very 
beginning of the kindergarten year. A notable addition in this current report is that results 
are now available on several other "non-cognitive" academic outcomes, including grade 
retention, attendance, recorded disciplinary actions, and special education services. 

Research design. There are several components to the overall research design for this 
evaluation. The component reported here, and the one that provides the strongest test of 
the effects of TN-VPK, is a randomized control trial in which children applying to TN-VPK 
are admitted to the program on a random basis. The TN-VPK programs participating in 
this part of the evaluation study were among those where more eligible children were 


1 That report, Evaluation of the Tennessee Voluntary Prekindergarten Program: End of Pre-K Results from 
the Randomized Control Design, is available on the Peabody Research Institute’s website (http://peabody. 
vanderbilt.edu/research/pri/). A copy of the Executive Summary is in Appendix A of the present report. 
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expected to apply for the program than there were seats available. Under such 
circumstances, only some applicants can be admitted and, of necessity, some must be 
turned away. The participating programs agreed to make this decision on the basis of 
chance, a process rather like randomly selecting names out of a hat, to determine which 
children would be admitted. This procedure treats every applicant equally and, as a result, 
no differences are expected on average between the characteristics of the children 
admitted and those not admitted. Comparing their academic and behavioral outcomes 
after the end of the pre-k school year, then, provides a direct indication of the effects of the 
TN-VPK program on the children who were admitted. 

To implement this procedure, TN-VPK programs across Tennessee that expected more 
applicants than they could accommodate and were willing to participate in the evaluation 
submitted lists of eligible applicants to the researchers at the Peabody Research Institute. 
The research team shuffled each list into a random order and the TN-VPK program staff 
were asked to fill the available seats by first offering admission to the child at the top of the 
list and then going down the list in order until all the available seats were filled. Once a 
program had admitted enough children to fill its seats, any remaining children were put on 
a waiting list and admitted, in order, if an additional seat became available. Those on the 
waiting list who were not admitted to TN-VPK became the control group for the study. 

This procedure was used for two cohorts of children, TN-VPK applicants for the 2009-10 
and 2010-11 school years, and resulted in more than 3000 randomly assigned children. 
Both the children who participated in TN-VPK and those who did not are being tracked 
through the state education database until their third grade year and information on 
various aspects of their academic performance and status is being collected each year. In 
addition, parental consent was obtained for a portion of this randomized sample, referred 
to as the Intensive Substudy. More than 1000 children in the Intensive Substudy are being 
directly assessed by the research team with a battery of early learning achievement 
measures, and are being rated by their teachers, in each year of the study. 

Outcome measures. The outcome measures used to assess the effects of TN-VPK were 
divided into two groups. One group consisted of measures of achievement in the areas of 
emergent literacy, language, and math that we refer to as cognitive achievement outcomes. 
The second group included measures of student performance or status other than 
academic achievement that we refer to as non-cognitive outcomes. This second group is 
especially relevant for assessing the longer term effects of TN-VPK because other 
longitudinal studies of early childhood education programs have found that effects on 
cognitive outcomes often fade after the end of the program while cumulative effects on 
non-cognitive outcomes emerge over time. 

Measures of Cognitive Achievement Outcomes. Academic gains of the children in the 
Intensive Substudy sample were measured with a selection of standardized tests from the 
Woodcock Johnson III Achievement Battery. These were individually administered at the 
beginning and end of the pre-k year, and one and two years afterwards when most of the 
children were nearing the end of their kindergarten and first grade years respectively. 
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These tests assessed early literacy, language, and math skills and included the following: 

Literacy 

• Letter-Word Identification-. Assesses the ability to identify and pronounce alphabet 
letters and read words. 

• Spelling : Assesses prewriting skills, such as drawing lines and tracing, writing letters, 
and spelling orally presented words. 

Language 

• Oral Comprehension-. Assesses children’s ability to fill in a missing word in a spoken 
sentence based on semantic and syntactic cues. 

• Picture Vocabulary-. Assesses early language and lexical knowledge by asking the child 
to name objects presented in pictures and point to the picture that goes with a word. 

• Passage Comprehension (not used in pre-k): Assesses reading comprehension through 
matching picture or text representations with similar semantic properties. 

Math 

• Applied Problems-. Assesses the ability to solve small numerical and spatial problems 
presented verbally with accompanying pictures of objects. 

• Quantitative Concepts: Assesses quantitative reasoning and math knowledge by asking 
the child to point to or state answers to questions on number identification, sequencing, 
shapes, symbols, and the like. 

• Calculation (not used in pre-k): Assesses mathematical computation skills through the 
completion of visually-presented numeric math problems. 

WI Composite 

• The scores on the above tests were summarized in two composite measures that 
averaged them together to create overall measures of children’s combined achievement 
in literacy, language, and math. One composite score combined the 6 tests given each 
year and the other also added the two tests given only in kindergarten and beyond. 

Measures of N on-Cognitive Outcomes. In addition, reports of the children’s work-related 

skills and behavior were obtained from their kindergarten teachers early in the fall of the 

school year after pre-k and from their first grade teachers near the end of the first grade 

year. Two teacher rating instruments were used for this purpose: 

• Cooper-Farran Behavioral Rating Scales: Teacher ratings for each child on two scales: 
o Work-Related Skills: The ability to work independently, listen to the teacher, 

remember and comply with instructions, complete tasks, function within designated 
time periods, and otherwise engage appropriately in classroom activities, 
o Social Behavior: Social interactions with peers including appropriate behavior 
while participating in group activities, play, and outdoor games; expression of 
feelings and ideas; and response to others’ mistakes or misfortunes. 

• Academic Classroom and Behavior Record: Teacher ratings for each child on three scales: 
o Readiness for Grade Level Work: How well prepared the child is for grade level work 

in literacy, language, and math skills as well as social behavior, 
o Liking for School: The child’s liking or disliking for school, having fun at school, 
enjoying and engaging in classroom activities, and seeming happy at school. 
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o Behavior Problems-. Whether the child has shown explosive or overactive behaviors, 
attention problems, physical or relational aggression, social withdrawal or anxiety, 
motor difficulties, and the like. 

o Peer Relations-. Whether other children in the classroom like the target child and 
how many close friends the target child has. 

Finally, data from the state education information system provided outcome measures for 
several additional aspects of student performance or status that might plausibly be affected 
by participation in TN-VPK: 

• Grade Retention: Whether a child was held back in a given grade, that is, not promoted 
to the next grade and thus required to repeat the grade they were in the previous year. 

• School Attendance: The total number of instructional days a child was marked as 
present summed across schools if a child was enrolled at more than one during a year. 

• Disciplinary Action-. Whether at least one disciplinary action was recorded in the state 
data system for a child in a given year. 

• Special Education Services-. Whether the state data system identified special education 
services for the child and the type of services indicated. 

Summary of Results Presented Earlier on the End ofPre-K Outcomes. The prior report 2 
presented findings from the Intensive Substudy portion of the randomized control design 
for outcomes at the end of the prekindergarten year. That sample included 1,076 children 
of families who applied to 58 TN-VPK programs across 21 school districts and was used to 
investigate two questions: 

1. Does participation in TN-VPK improve the school readiness of the economically 
disadvantaged children eligible for the program? 

2. What are the characteristics of the children who benefit the most from TN-VPK? 

All the children in the Intensive Substudy sample qualified for the federal Free or Reduced 
Price Lunch program and those who participated in TN-VPK attended an average of 149 
days during the school year. In contrast, more than half of the children who were not 
admitted to TN-VPK stayed home with a parent or other guardian and 27% were enrolled 
in Head Start or private center-based childcare. 

During the course of the pre-k school year, the academic skills of all the children improved. 
However, the children who participated in TN-VPK gained significantly more on all the 
direct assessments of academic skills than the children who did not attend. The effect size 
for the WJ Composite scale was .24, and the effect sizes for the individual literacy, language, 
and math scales ranged from .10 to .46. Stated in terms of the gains made on these 
measures during the pre-k year, the children who participated in TN-VPK showed a gain on 
the WJ Composite measure that was 45% greater than that made by the children who did 
not attend TN-VPK. On the individual academic achievement measures, the analogous 
improvements for the TN-VPK participants relative to the nonparticipants ranged from 
21% to 89%. Positive effects were also found on the kindergarten teachers’ ratings of 
children’s preparedness for kindergarten and, to a lesser extent, on their ratings of the 
children’s classroom work behavior and social behavior. These effects were not different 


2 See footnote 1. 
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for boys and girls, but there were larger effects on the academic skills of children who were 
not native English speakers than for those who were. 

The stated goal of TN-VPK is to increase the school readiness of the economically 
disadvantaged children it serves. These findings show that the Tennessee prekindergarten 
program is successful in producing significant improvements in a range of academic skills 
generally regarded as important for school readiness for the children who participate. 

Summary of Results on the End of Kindergarten and First Grade Outcomes. This report 
presents findings for TN-VPK effects on kindergarten outcomes and on those first grade 
outcomes for which data are available to date for both the full randomized sample 
(N=3025) and the Intensive Substudy sample (N=1076). The analyses addressed three 
questions about what effects of TN-VPK are evident one and two years after children have 
participated in the program. Below we summarize the findings for each of those questions. 

Does participation in TN-VPK have effects on children's cognitive achievement outcomes 

that are evident at the end of the kindergarten and first grade years? 

Cognitive achievement outcomes were measured near the ends of the kindergarten and 
first grade years with Woodcock Johnson III scales selected to assess performance in the 
areas of literacy (Letter-Word Identification and Spelling), language (Oral Comprehension, 
Picture Vocabulary, and Passage Comprehension), and math (Applied Problems, 
Quantitative Concepts, and Calculation). Because administering these measures required 
parental consent and individual assessment of the children by the research team, data were 
available only for the Intensive Substudy sample. 

For that Intensive Substudy sample, we found that the effects of TN-VPK on the WJ 
achievement measures observed at the end of the pre-k year had greatly diminished by the 
end of the kindergarten year and the differences between participants and nonparticipants 
were no longer statistically significant. The only exception was a marginally significant 
difference on Passage Comprehension with nonparticipants showing higher scores at the 
end of the kindergarten year than TN-VPK participants. 

Similarly, at the end of first grade, there were no statistically significant differences 
between TN-VPK participants and nonparticipants on the WJ measures with one exception. 
There was a significant difference that favored the nonparticipant group on the 
Quantitative Concepts subscale. 

These diminished effects were not entirely unexpected in light of the findings in other 
longitudinal studies of the effects of early childhood programs on economically 
disadvantaged children. For preschool programs, a typical finding is that the cognitive 
effects are not sustained for very long after the end of the program. Though none of those 
other studies investigated the effects of a single year of a scaled up state-funded public pre- 
k program, many involved more intensive programs that nonetheless also failed to show 
long-term effects on cognitive achievement measures. It should be noted that few of the 
programs, including TN-VPK, involved continuous, focused support in subsequent years for 
the gains made during the initial program year. 

The especially rapid fall off of TN-VPK effects is somewhat surprising, however, and raises 
questions about why it occurs that cannot be answered directly by the data available in the 
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current study. There is reason to believe from prior research that early cognitive gains 
attained in pre-k can be sustained for a longer period if they are large enough to begin with 
and/or continuously supported with effective instruction in subsequent years. 

Does participation in TN-VPK have effects on children's non-cognitive academic outcomes 

that are evident at the end of the kindergarten and first grade years? 

Expectations for TN-VPK effects on these non-cognitive outcomes were different than for 
the cognitive achievement outcomes. Whereas longitudinal research on other early 
education programs provides little evidence of sustained effects on achievement, they 
show more encouraging effects on non-cognitive outcomes. However, it is important to 
note that those effects typically have been reported for follow-up periods that included 
many years after the initial program and thus seem to be more cumulative than immediate. 

First Grade Teachers' Ratings. One set of measures of non-cognitive outcomes for TN-VPK 
was available from ratings made for the children in the Intensive Substudy sample by first 
grade teachers near the end of the first grade year. Those showed no statistically 
significant differences between the TN-VPK participants and nonparticipants on any of the 
scales for Social Skills, Work-related Skills, Preparation for Grade, Peer Relations, Behavior 
Problems, and Feelings About School. 

Another set of measures of non-cognitive outcomes was available from the state data 
system for the Intensive Substudy sample and at least Cohort 1 of the full randomized 
sample, with some measures available for both cohorts. These outcomes included grade 
retention in kindergarten and attendance, officially recorded disciplinary actions, and 
identified special education services in kindergarten and first grade. 

Retention. For the Intensive Substudy sample (N=1076), there was a statistically significant 
difference between the 4.1% of the TN-VPK participants retained in kindergarten 
compared to the 6.2% retention rate for the nonparticipants. This effect was confirmed in 
Cohort 1 of the full randomized sample, with retention data still unavailable for Cohort 2. 

In Cohort 1 (N=1764), 4.0% of the TN-VPK participants were retained in kindergarten 
compared to 8.0% of the nonparticipants. 

Attendance. Attendance in kindergarten and first grade was high for both TN-VPK 
participants and nonparticipants, so there was little room for large differences. For the 
Intensive Substudy sample the difference in days attended between participants and 
nonparticipants was not statistically significant. In the full randomized sample, however, 
there was a marginally significant effect on the number of days attended that favored the 
TN-VPK participants, though the difference was small — less than two days of attendance. 

Attendance data for first grade were available only for Cohort 1 of the full randomized 
sample. For that cohort, attendance was about three and a half days greater for the TN- 
VPK participants than the nonparticipants, a statistically significant difference. This effect 
was thus somewhat larger than the one found for kindergarten attendance. 

Disciplinary Actions. Officially recorded disciplinary actions were very infrequent during 
the kindergarten and first grade years, appearing for less than 1.5% of the children in 
kindergarten and less than 3% in first grade. These low rates thus provide little scope for 
meaningful differences between TN-VPK participants and nonparticipants. During 
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kindergarten, there were no significant differences in the percentages of children with any 
recorded disciplinary action in the Intensive Substudy sample or the full randomized 
sample. For first grade, data were available only for Cohort 1 of the full randomized sample 
and also showed no significant difference between TN-VPK participants and 
nonparticipants. 

Special Education Services. The final non-cognitive outcome examined in this report was 
recorded special education services. Only descriptive results are presented for that 
outcome because of the multiplicity of special education designations and ambiguity about 
how to interpret the identification of the children in our samples for special education 
services. TN-VPK participants had an opportunity to have their special needs identified 
and addressed during the pre-k year that was not afforded to the nonparticipants. 
Moreover, having that earlier year means those children are more likely to receive 
continued services in the kindergarten and first grade years. Arguably, this is a positive 
outcome — earlier identification and services for special needs for the TN-VPK participants 
should reduce their need for such services in later grades. On the other hand, if TN-VPK 
participants should need and receive more special education services over a long term, that 
would not be a positive outcome. The number of years that constitutes that long term is 
not clear, but we do not believe that two years after pre-k is sufficient to determine how 
special education needs and services will unfold as a result of participation in TN-VPK. 

The findings so far, as expected, show that TN-VPK participants received more special 
education services in kindergarten than nonparticipants, though the difference was smaller 
when only services for needs other than the common speech/language problems were 
considered. However, the difference was reversed when only new special education 
designations were examined. Somewhat fewer TN-VPK participants had first designations 
for a special education service in kindergarten than nonparticipants. This too is not 
surprising; kindergarten is the first opportunity for the special needs of nonparticipants to 
be identified and addressed. 

First grade special education data were available only for Cohort 1 of the full randomized 
sample. As with the kindergarten comparison, they showed a greater percentage of TN- 
VPK participants receiving special education services than nonparticipants, but with much 
smaller percentages for both when speech/language services are not counted. 

The non-cognitive effects of TN-VPK are important because of their potential long-term 
influence on children’s academic careers and the findings in other studies of early 
childhood education that show this to be the domain in which the largest effects occur. 
Further, the literature identifies these non-cognitive outcomes as those with the biggest 
cost saving implications for schools and communities. It is too early to expect such effects 
to appear with any consequential magnitude for TN-VPK, but there are early promising 
signs in the positive findings for kindergarten grade retention and first grade attendance. 
The further follow-up waves planned through at least third grade will, in time, provide a 
fuller picture of TN-VPK effects on these non-cognitive academic outcomes. 
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What are the characteristics of the children who show the largest effects of participation in 

TN-VPK at the end of the kindergarten and first grade years? 

No differential TN-VPK effects on the WJ composite achievement measures were found for 
gender, age, or native English speaker subgroups after either the kindergarten or first 
grade year. There were thus no strong overall effects of TN-VPK for any of these subgroups 
when all of the achievement measures were combined into a single composite. 

Nonetheless, there were some indications of differential TN-VPK effects in particular 
achievement domains. For participants who entered pre-k when they were younger, TN- 
VPK had greater effects on the literacy measures (Spelling and Letter-Word Identification) 
and one of the math measures (Calculation) in kindergarten with similar differential effects 
for two of the math measures (Applied Problems and Calculation) in first grade. An age 
differential also was found for the first grade teachers’ ratings. According to those 
teachers, the younger children benefitted more from TN-VPK than their older counterparts 
on work-related skills. In contrast, they reported larger effects of TN-VPK on social skills 
for the older children. 

Similarly, the effects of TN-VPK in kindergarten and first grade on some of the measures of 
literacy and language achievement were stronger for native English speaking children than 
for the non-native English speakers, though it is the latter that most need a boost in those 
skills. This pattern appeared in the Letter-Word identification measure in kindergarten 
and more strongly in two of the language measures (Picture Vocabulary and Passage 
Comprehension) in first grade. These findings are a reversal of the pattern of results at the 
end of the pre-k year — the non-native English speaking children showed larger TN-VPK 
effects than the native English speaking children at that time on all the achievement 
measures. In light of this reversal, it may not be surprising that participation in TN-VPK 
had less positive effects on feelings about school, as rated by the first grade teachers, for 
the non-native English speakers than for the native English speakers. 

There were also some indications of differential TN-VPK effects on kindergarten retention. 
TN-VPK had larger effects (less retention) for native English speaking children than for 
non-native English speakers. This differential was statistically significant in the analysis 
with the full randomized sample and in the same direction, but not significant in the 
Intensive Substudy sample. A similar pattern was evident at a marginally significant level 
for TN-VPK effects on kindergarten attendance — native English speaking children showed 
larger effects than non-native English speaking children. That same pattern of differential 
effects appeared for first grade attendance but, again, was only marginally significant. 

Concluding Note L The longitudinal effects found for TN-VPK so far are decidedly mixed. 

The encouraging achievement effects found at the end of the pre-k year were not sustained, 
but there are indications of possible effects on important non-cognitive academic 
outcomes. The kindergarten and first grade years are too early for any such effects to 
appear in anything but tentative form, however, so later waves of data will be required to 
paint the full picture of those effects. Also ahead are the state achievement tests that are 
administered in third grade with an open question of whether TN-VPK participation will 
have any effect on those critical performance measures. 
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